Download resources from osf #277
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TODO: use other urls too
- HIPPUNFOLD_CACHE_DIR now has model, atlas, and template subdirs
avoids breaking the seg* modalities
Closed
|
Just realized I forgot the upenn template, will do that now |
|
actually, I'll leave tpl-upenn there for now, as the shape-inject stuff gets edited with the template-based segmentation edits which will be in another PR.. |
|
Very nice! Perhaps this would be a good branch to scrub the large files (i.e. mostly resources) from our git history to reduce the size of clones? |
|
I think that would be done at the git repository level since it would scrub things beyond this branch.. but I'll have a look now |
akhanf
added
the
maintenance
akhanf
added a commit
that referenced
this pull request
Feb 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a download.smk rules file and a couple of rules to download+extract zip files from osf for templates and atlases. Will go in template and atlas subfolders in the hippunfold cache dir (ie HIPPUNFOLD_CACHE_DIR, or appdir's cache which defaults to the home directory).
The files are extracted using a snakemake output directory(), so the contents are not monitored by snakemake, only the folders. So for downstream rules the files have to be specified in params, using the folder (ie atlas_dir, template_dir) in the inputs. Only downside is a dry-run without files downloaded won't be able to check to see if the
config['template_files']orconfig['atlas_files']entries are correct. But could just dohippunfold --until download_extract_atlas_or_template -c 1to pre-download the relevant resources, then a dry-run after that would work..Note: the folder structure of the HIPPUNFOLD_CACHE_DIR will change since models won't be in the top level. We could document this perhaps in the next release notes to indicate how to move them: